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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

• If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment See 37 CFR 1 .704(b). 

Status 

1 )S Responsive to communication(s) filed on 25 February 2004 . 
2a)M This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) S Claim(s) 1-3,5-8 and 15-21 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) S Claim(s) 15-21 is/are allowed. 

6) S Claim (s) 1-3 and 5-8 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

1 1) D The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 119 and 120 

13) H Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)l3AII □)□ Some*c)D None of: 

1 .13 Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 

Attachments) 

1) M Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s). . 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-1 52) 

3) I3 Information Disclosure Statement(s) (PTO-1449) Paper No(s) 13. 6) □ Other: 
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DETAILED ACTION 



1 . This action is responsive to communications: Amendment, filed on 2/25/2004. 
This action is final. 

2. Claims 1-3, 5-8 and 15-21 are pending in this application. Claims 1, 15, 16 and 
18-21 are independent claims. 

This application claims foreign priority dated 6/1 1/99. 

3. The present title of the invention is "Marker Layout Method, Mixed Reality 
Apparatus, and Mixed Reality Space Image Generation Method" as filed originally. 



4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 1-3 and 5-7 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Cohen et al. (6,222,937) and further in view of Hirota et al. (6,064,749) 

As per claim 1, Cohen et al., hereinafter Cohen, disclose a marker layout method 
for laying out markers in a real space as position indices upon presenting a mixed 
reality space shared by a plurality of players, each of which is movable within a different 
movable range, wherein the mixed reality space is generated by mixing a real space 
and virtual space, comprising the step of: 



Claim Rejections - 35 (JSC § 103 
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laying out the markers to have a positional relationship that allows a given player 

to observe only markers that are needed by the given player when plurality of players 

observe the mixed reality space within their respective different movable ranges ("The 

walls and base of the stage are inscribed with markers 605", column 7, line 18-19, and 

""various pre-defined vantage points" (column 7, line 50) are the observe points), 

wherein markers to be used by only the given player are laid out at positions 
hidden by real objects when the markers are observed from the other players ("The 
collection system automatically determines the vantage point of a picture by (1 ) 
identifying those markers not occluded by the object", column 7, line 24-26, thus, some 
markers hidden from some view points is observable from some other view points). 

Cohen discloses a marker layout method where some markers are hidden from 
some viewers. It is noted that Cohen does not explicitly disclose wherein the markers 
are used to detect viewpoint positions of the plurality of players, however, this is known 
in the art as taught by Hirota et al., hereinafter Hirota. Hirota discloses a tracking 
method in augmented reality in which "landmarks is utilized as the primary method for 
determining camera position and orientation" (column 3, line 16-18). 

Thus, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to incorporate the teaching of Hirota into Cohen because Cohen 
discloses a marker layout method and Hirota discloses the viewer positions can be 
determined from the markers in order to provide accurate registration. 
6. As per claim 2, Cohen demonstrated all the elements as applied to the rejected 
claim 1, supra, and further discloses a visible feature of the markers to be used by only 
the other players is similar to a visible feature of the markers to be used by only the 
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given player (Figure 6, since the different players share the same processor and 

memory, it is anticipated the marker used are similar). 

7. As per claim 3, Cohen demonstrated all the elements as applied to the rejected 
claim 1 , supra, and further discloses the visible feature includes at least one of color, 
texture, shape, and size of the marker ("markers 605 colored a darker shade of cyan", 
column 7, line 18-19). 

8. As per claim 5, Cohen demonstrated all the elements as applied to the rejected 
claim 1 , supra, and further discloses the real objects are laid out for an application that 
uses the mixed reality space (since Lumigraph system collects a synthetic or real object 
(or a scene), see Abstract). 

9. As per claim 6, Cohen demonstrated all the elements as applied to the rejected 
claim 1 , supra, and further discloses the markers to be laid out include markers shared 
by a plurality of players (since the markers are observable in a 3-D space, the markers 
not obstruct by objects can also be observed by some other players). 

10. As per claim 7, Cohen demonstrated all the elements as applied to the rejected 
claim 1 , supra, and further discloses the markers have a common color ("markers 605 
colored a darker shade of cyan", column 7, line 18-19, it would have been obvious to 
one of ordinary skill in the art to make the marker of the same color in order to make 
them to be recognizable). 

11. As per claim 8, Cohen demonstrated all the elements as applied to the rejected 
claim 1, supra, and further discloses computing and/or correcting location/posture 
information of a player using markers laid out by a marker layout method of claim 1 
(Figure'21 2105). 
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Allowable Subject Matter 

12. Claims 15-21 are allowed. 

As per claims 15-21 , the closest prior art by Cohen et al. do not explicitly disclose 
in a mix reality space image generation, a marker layout method the step of 

"A mixed reality space image generation step of generating a mixed reality space 
image to be observed by the player, so the player observes virtual object images that do 
not include any images of markers in marker regions including the markers from the 
image data, 

wherein said mixed reality space image generation step includes a step of 
substituting or overlaying images of the marker regions by predetermined virtual object 
images". 

Any comments considered necessary by applicant must be submitted no later 
than the payment of the issue fee and, to avoid processing delays, should preferably 
accompany the issue fee. Such submissions should be clearly labeled "Comments on 
Statement of Reasons for Allowance." 

Response to Arguments 

13. Applicant's arguments with respect to claims 1-3 and 5-8 have been considered 
but are moot in view of the new ground(s) of rejection. 

Conclusion 

14. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 
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15. Applicant's amendment necessitated the new ground(s) of rejection presented in 

this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 

CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Inquiries 

1 6. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ryan Yang whose telephone number is (703) 308- 
6133. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Michael Razavi, can be reached at (703) 305-4713. 

Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 
Washington, D.C. 20231 
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or faxed to: 

(703) 872-9314 (for Technology Center 2600 only) 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal 
Drive, Arlington, VA, Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the Technology Center 2600 Customer Service Office 
whose telephone number is (703) 305-47000377. 



Ryan Yang 
April 25, 2004 




